1. Technical Field
The invention generally relates to the area of human-computer interaction, and more particularly relates to self-contained inertial navigation systems (INS) for interactive control using movable controllers in applications such as video games. With one embodiment of the present invention, inputs from a hand-held controller containing inertial sensors allow a user to control an object within a two or three dimensional representation shown to the user, and allow the user to directly manipulate one or more chosen objects by mapping the positions of the controller in a physical world to corresponding positions in a virtual world. Various embodiments of the present invention may be used in computer applications, video games, or on-screen controls for electronic devices.
2. Related Art
There are a number of man-machine interface devices, such as computer mice, joysticks, remote controllers and trackballs, for controlling computer applications and video games. Each of these devices are well understood in the art and primarily focus on converting motions of a human being into an analog motion that is represented on a two dimensional screen. For example, a joystick translates the position of the control stick relative to a center into a velocity that is applied to a point located on a two dimension screen. According to an established convention, a left-right motion of the control stick corresponds to left-right motion on the screen and a forward-away motion of the control stick corresponds to up-down motion on the screen.
This basic approach of remapping inputs to control motions has been extended to cover three-dimensional computer applications using objects such as 3D mice and 3D joysticks. One approach of doing this is described in U.S. Pat. No. 5,898,421. Most of these approaches available today, however, have the disadvantage that the users must learn an artificial convention for how their motions in the physical world correspond to the motions of a pointer in a computer representation. In general, users prefer natural interactions with a computer application.
A natural interaction for a user would be to have a direct control over the motion of an object in a displayed scene. For example, in a sword-fighting game, a natural control for the user would be to have the sword displayed in the game with the same orientation and position as the motion controller in his/her hand. Currently this is possible by having an external system that measures the exact position and/or orientation of the controller in the physical world. A system for doing this is described in U.S. Pat. No. 4,862,152 but requires the addition of bulky sonic sensors and emitters in the vicinity of the user. Essentially, the system is limited by restricting the motions of the user within a predefined range.
Another natural interaction a user desires is the ability to directly point at objects by using his/her hand to point at the image being shown on the display. A two-dimensional solution to this particular style of interaction is introduced by Nintendo in the Wii system (US Patent Publication No.: US20070060384), however it requires additional modification of the environment of the user by adding a sensor bar to define a limited range and field of view, thus restricting the movements of the user to a small range in front of the sensor bar. It would be desirable to have an approach that requires less modification to the user's environment and allows natural three-dimensional pointing interactions.
There is thus a need for techniques to track effectively a controller(s) when the controller(s) is being manipulated in a physical space in reacting to a virtual space being displayed. Such techniques provide a full control on motions of displayed objects in six degrees of freedom. Such techniques shall work in situations in which there are no additional sensors or emitters around the physical space. When there are additional sensors or emitters around the physical space, such techniques are provided to utilize the external sensing data to track the controller with minimize errors.